Imaging apparatus and image restoration method

ABSTRACT

An imaging apparatus restores a deteriorated image to a high-resolution image when the deteriorated image is restored based on a PSF image captured by an optical system. An imaging apparatus includes an optical system, and a PSF capturing unit that acquires point spread function (PSF) information captured by the optical system and outputs corrected PSF information. A subject capturing unit acquires subject information captured by the optical system and outputs the subject information; and an image restoration unit performs a restore operation for restoring the subject information, based on the corrected PSF information and the subject information. The PSF capturing unit subtracts a correction luminance value from the PSF information, and outputs the corrected PSF information. The correction luminance value is greater by a luminance value Is than a luminance value Nf of fixed value noise that does not fluctuate with time.

TECHNICAL FIELD

The present invention relates to technology for restoring an imagedeteriorated when the image was captured to an image that is lessdeteriorated.

BACKGROUND OF INVENTION Background Art

Techniques for restoring an image deteriorated when the image wascaptured due to a factor such as defocus, blur, or aberration of anoptical system to an image that is less deteriorated have beenprogressively developed. For example, with the technique disclosed inPatent Literature 1, it is possible to obtain a restored image bycorrecting deterioration of a deteriorated image (captured image) thatis deteriorated due to defocus, blur, aberration, or the like inaccordance with a restore operation using a correction function havinginverse characteristics of those of a point spread function (PSF)resulting from defocus, blur, aberration, or the like. In many cases,such correction functions are created using PSF data created by acomputer based on design data or the like.

Further, with the technique disclosed in Patent Literature 2, when it isdifficult to create PSF data, a restore operation for restoring adeteriorated image is performed using PSF data obtained by actualshooting.

CITATION LIST Patent Literature

-   [PTL 1] Japanese Unexamined Patent Application Publication No.    62-127976-   [PTL 2] Japanese Unexamined Patent Application Publication No.    2009-163642

SUMMARY OF INVENTION

However, in the case where a restore operation for restoring adeteriorated image is performed using PSF data created by a computerbased on design data or the like, a high-resolution restored imagecannot be obtained when there is a big difference between a PSFindicated by the PSF data and the actual PSF due to, for instance, alarge mounting error being generated when the camera is assembled. Thus,it may be necessary to perform image restoration using a PSF imageobtained by actual shooting, rather than using the PSF data created bythe computer.

Also, in the case where a restore operation for restoring a deterioratedimage is performed using a PSF image obtained by shooting a point lightsource, rather than using PSF data created by a computer as in PatentLiterature 2, especially if unnecessary luminance (hereinafter,described as “noise” as appropriate) of an imaging device is high whenthe PSF image is captured, the PSF shown by the PSF image differs fromthe actual PSF. Consequently, this results in a problem that ahigh-resolution restored image cannot be obtained.

In view of this, the present invention has been conceived to solve theabove problems, and an object thereof is to provide an imaging apparatusand an image restoration method that enable restoration of adeteriorated image to a high-resolution image when the deterioratedimage is restored based on a PSF image captured by an optical system.

In order to achieve the above object, an imaging apparatus according toan aspect of the present invention includes: an optical system; a PSFcapturing unit configured to acquire point spread function (PSF)information captured by the optical system, and output corrected PSFinformation; a subject capturing unit configured to acquire subjectinformation captured by the optical system, and output the acquiredsubject information; and an image restoration unit configured to performa restore operation for restoring the subject information, based on thecorrected PSF information and the subject information, wherein the PSFcapturing unit is configured to subtract a correction luminance valuefrom the PSF information, and output the corrected PSF informationobtained as a result of the subtraction, the correction luminance valuebeing greater by a luminance value Is than a luminance value Nf of fixedvalue noise that does not fluctuate with time.

In this manner, it is possible to reduce the influence of random noiseincluded in the PSF information captured by the optical system bysubtracting the luminance value greater than the fixed value noise fromthe PSF information. Consequently, this enables restoration of adeteriorated image to a high-resolution image.

According to an imaging apparatus according to an aspect of the presentinvention, when an image restore operation is performed, even in thecase where unnecessary luminance (especially, random noise thatfluctuates with time) of a captured PSF image is high, the unnecessaryluminance is reduced so as to correct a luminance average value to anappropriate value, thereby obtaining more accurate restorationinformation and enabling high-resolution image restoration.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a configuration of an imagingapparatus according to Embodiments 1 and 2 of the present invention.

FIG. 2 illustrates the relationship among an original image, a PSFimage, and a deteriorated image in the embodiments of the presentinvention.

FIG. 3 shows a PSF luminance distribution in the embodiments of thepresent invention.

FIG. 4 shows a simulator for verifying the influence exerted by noise ona restored image in the embodiments of the present invention.

FIG. 5 illustrates the relationship between noise included in adeteriorated image and a restored image in the embodiments of thepresent invention.

FIG. 6 illustrates the relationship between noise included in a PSFimage and a restored image in the embodiments of the present invention.

FIG. 7 illustrates the relationship between a resolution of a restoredimage and noise included in a deteriorated image and a PSF image in theembodiments of the present invention.

FIG. 8 shows ideal PSF information in the embodiments of the presentinvention.

FIG. 9 shows PSF information including noise in the embodiments of thepresent invention.

FIG. 10 shows flowcharts showing the operation of the imaging apparatusaccording to Embodiment 1 of the present invention.

FIG. 11 shows PSF information in Embodiment 1 of the present invention.

FIG. 12 shows PSF information in Embodiment 1 of the present invention.

FIG. 13 shows restored images in Embodiment 1 of the present invention.

FIG. 14 shows the resolution of restored images in Embodiment 1 of thepresent invention.

FIG. 15 shows flowcharts showing operation of the imaging apparatusaccording to Embodiment 2 of the present invention.

FIG. 16 shows PSF information in Embodiment 2 of the present invention.

FIG. 17 shows restored images in Embodiment 2 of the present invention.

FIG. 18 shows the resolution of restored images in Embodiment 2 of thepresent invention.

DETAILED DESCRIPTION OF INVENTION

The following is a description of a factor that makes high-resolutionimage restoration difficult when unnecessary luminance (noise) of acaptured PSF image is high, before giving a description of embodimentsof the present invention.

A factor that makes high-resolution image restoration difficult whennoise of a captured PSF image is large will now be described withreference to FIGS. 2 to 9. (a) in FIG. 2 shows an original image(subject) having no deterioration. The image is a cuneal chart generallyused when the resolution of a captured image is measured. (b) in FIG. 2shows an example of a PSF image captured by an optical system.

Due to, for instance, defocus, blur, or aberration of the opticalsystem, a point image has finite spread as shown in (b) in FIG. 2.Accordingly, the original image in (a) in FIG. 2 will be formed on animaging device via the optical system, as a deteriorated image having alower resolution as shown in (c) in FIG. 2. It is known that adeteriorated image is expressed using convolution integral of anoriginal image and a PSF image that has been normalized such that aluminance integral value of the entire image region is “1”.

It should be noted that FIG. 3 shows a luminance distribution in anenlarged periphery of a region having the highest luminance on linesthat include a portion having the highest luminance of the PSF image in(b) in FIG. 2. FIG. 3 shows a luminance distribution when the PSF imageincludes no noise. Although the expression of the luminance of an imagediffers depending on a system on which the present invention is mounted,“0” represents black, and “1.0” represents white, here.

FIG. 4 is a block diagram showing a simulator for examining theinfluence exerted by noise mixed in a deteriorated image and that in aPSF image when a subject image formed on the imaging device is captured.The simulator includes a deteriorated-image noise adding unit 101 thatadds noise to a deteriorated image including no noise, a PSF-image noiseadding unit 102 that adds noise to a PSF image including no noise, andan image restore operation unit 103. It is possible to examine theinfluence of noise exerted on each of the deteriorated image and the PSFimage using this simulator.

Noise is assumed to be Gaussian noise. The influence of the noise can beexamined by changing the standard deviation σ of the Gaussian noise. Forexample, if noise values at respective image positions are obtained inadvance, this enables compensation, with ease, of fixed value noise thathardly changes with time depending on image positions (e.g., darkcurrent noise, noise that occurs in predetermined lines or atpredetermined pixel positions due to manufacturing defects of theimaging device, or the like), and thus such fixed value noise is nottaken into consideration here. Specifically, the influence of noise isexamined, taking into consideration only random noise (assumed to beGaussian noise) that randomly changes with time and that is difficult tobe compensated. It should be noted that an image position is a positionon an image, and is typically a position of a pixel that constitutes theimage. Further, Gaussian noise is noise in which the distribution ofluminance values of noise components approximates a Gaussiandistribution.

The image restore operation unit 103 may perform an image restoreoperation using an algorism known as an image restore algorithm; such asthe Wiener filter or the Richardson-Lucy algorithm. Here, the imagerestore operation unit 103 has a configuration of obtaining a restoredimage by performing an image restore operation using the Wiener filter.

For the configuration of the Wiener filter Hw(u, v), Expression 1 below,for example, may be used, which is described in Non Patent Literature(Digital Image Processing: CG-ARTS Society, Jul. 22, 2004, p. 146).Hw(u,v)=1/H(u,v)·|H(u,v)|^2/(|H(u,v)|^2+K)  (Expression 1)

Here, H(u, v) represents an optical transfer function (OTF) that is theFourier transform of a PSF image. Further, “u” represents the address ofan array where frequency components in the vertical direction of the PSFimage are stored. Also, “v” represents the address of an array wherefrequency components in the horizontal direction of the PSF image arestored. “K” is an appropriate constant.

The image restore operation unit 103 multiplies Fourier transform dataof a deteriorated image by the Wiener filter Hw(u, v) for each frequencycomponent, and generates a restored image by performing inverse Fouriertransform on the multiplication results. The cuneal chart shown in (a)in FIG. 2 is used as a subject. The PSF-image noise adding unit 102 addsnoise to the PSF image in (b) in FIG. 2 as necessary, and thereafter theobtained image is normalized such that the luminance integral value ofthe entire region is 1. The resultant image is used as a PSF image.

(a), (b), and (c) in FIG. 5 show restored images respectivelycorresponding to standard deviations σ in the case where thedeteriorated-image noise adding unit 101 adds Gaussian noise whosestandard deviation σ has been changed to the deteriorated image in (c)in FIG. 2. The PSF image and the deteriorated image are 512×512 pixelimages, as examples.

Specifically, (a), (b), and (c) in FIG. 5 show restored images in thecase where the standard deviation σ is set to 0%, 0.05%, and 0.3% of thehighest luminance setting value (here, the highest luminance settingvalue is “1” since “0” indicates black, and “1.0” indicates white) ofthe deteriorated image. Noise is not added to the PSF image at thistime. As is clear from (a), (b), and (c) in FIG. 5, the resolution ofthe restored images is slightly decreased as the standard deviation σ isincreased. It should be noted that the highest luminance setting valueis a value indicating the highest luminance among values that eachindicate a luminance.

(a), (b), and (c) in FIG. 6 show restored images respectivelycorresponding to standard deviations σ in the case where the PSF-imagenoise adding unit 102 adds Gaussian noise whose standard deviation σ hasbeen changed to the PSF image in (b) in FIG. 2. Specifically, (a), (b),and (c) in FIG. 6 show restored images in the case where the standarddeviation σ of Gaussian noise is set to 0%, 0.05%, and 0.3% of thehighest luminance value of the PSF image. Noise is not added to thedeteriorated image at this time. As is clear from (a), (b), and (c) inFIG. 6, the resolution of the restored images is significantly decreasedas the standard deviation σ is increased.

It should be noted that the highest luminance value of the PSF image isa luminance value at an image position that shows the highest luminancein the PSF image. Specifically, the highest luminance value of the PSFimage is a luminance value of a pixel that shows the highest luminanceamong pixels that constitute the PSF image, for example.

FIG. 7 shows the result of comparison of the change in the resolution ofrestored images in the case where the standard deviation σ of Gaussiannoise is changed. In FIG. 7, numeral 701 denotes the resolution of arestored image in the case where Gaussian noise is added only to adeteriorated image. Further, numeral 702 denotes the resolution of arestored image in the case where Gaussian noise is added only to a PSFimage.

The standard deviation σ of Gaussian noise to be added to thedeteriorated image is represented by the proportion to the highestluminance setting value. Further, the standard deviation σ of Gaussiannoise to be added to the PSF image is represented by the proportion tothe highest luminance value. Here, when Gaussian noise is added to thePSF image, the PSF image has been normalized such that the highestluminance value is the highest luminance setting value “1.0”, andcomparison is performed on the condition that the same noise is added tothe deteriorated image and the PSF image.

The resolution is measured using the resolution measurement toolHYRes3.1 distributed by CIPA, with reference to CIPA DC-003 “ResolutionMeasurement Methods for Digital Cameras”. The resolution measured inthis manner indicates that the more lines are measured, the higher theresolution is. It should be noted that in the embodiments of the presentinvention, the number of lines of resolution of a restored image that isrestored in the case where Gaussian noise is not added to either thedeteriorated image or the PSF image is 428.

As is clear from FIG. 7, the resolution of a restored image changesdepending on the ratio between an image signal and the standarddeviation σ of Gaussian noise. Further, it can be seen that when thestandard deviation σ of Gaussian noise is increased, the resolution ofthe PSF image more significantly decreases compared with that of thedeteriorated image.

In the case where Gaussian noise is added only to the deterioratedimage, when the standard deviation σ of Gaussian noise is equal to orgreater than 0.6% of the highest luminance setting value, the resolutionsignificantly decreases, and the number of lines of resolution becomes 0(which cannot be measured). On the other hand, in the case whereGaussian noise is added only to the PSF image, when the standarddeviation σ of Gaussian noise is equal to or greater than 0.3% of thehighest luminance value, the resolution significantly decreases, and thenumber of lines of resolution becomes 0.

Consequently, it has been found that noise included in the PSF image hasgreater negative influence on a restored image than the noise includedin the deteriorated image. Note that when the resolution of thedeteriorated image in (c) in FIG. 2, which has not been restored, ismeasured, the result will indicate that measurement cannot be performedbecause of the influence of blur due to aberration, regardless ofwhether noise is included, and the number of lines of the resolution is0.

The result obtained by verifying the factor of a great decrease in theresolution of a restored image due to noise included in the PSF imagewill be described with reference to FIGS. 8 and 9.

(a) in FIG. 8 shows a luminance distribution of enlarged lines in theperiphery of a portion having the highest luminance of the PSF image in(b) in FIG. 2. Here, the PSF image does not include noise.

(b) in FIG. 8 shows the gain of an OTF that is the Fourier transform ofthe PSF image that does not include noise. The OTF in (b) in FIG. 8 hasbeen normalized such that the gain of the direct-current component (at afrequency of 0) is 1. The horizontal axis in (b) in FIG. 8 representsfrequency, and the right side thereof relative to the direct-currentcomponent (at a frequency of 0) represents positive frequency, whereasthe left side thereof represents negative frequency. For the PSF imagein (b) in FIG. 2, an example is used in which a luminance distributionis symmetrical with the image position having the highest luminancebeing the center. In view this, to simplify the gain distribution, in(b) in FIG. 8 and the following drawings that show an OTF, one line'sworth data including direct-current component data in the vertical andhorizontal directions in the two-dimensionally arrayed OTF is extractedand displayed.

(a) in FIG. 9 shows a luminance distribution of enlarged lines in theperiphery of a portion having the highest luminance of an image obtainedby adding Gaussian noise whose standard deviation σ is 0.3% of thehighest luminance value to the PSF image in (b) in FIG. 2. (b) in FIG. 9shows the gain of an OTF that is the Fourier transform of the PSF imagethat includes this noise. The OTF in (b) in FIG. 9 has also beennormalized such that the gain of the direct-current component (at afrequency of 0) is 1.

Compared with (b) in FIG. 8, it can be seen from (b) in FIG. 9 that thegain of a component at a frequency of 0 (direct-current component) ismuch greater compared with the gain of other frequency components. Aconceivable reason for this is that most of the entire PSF image in (b)in FIG. 2 is a region having a low luminance, and the addition of noiseto that low-luminance region greatly changes the average luminance value(=direct-current component) of the entire PSF image.

Therefore, the resolution of a restored image is greatly decreased dueto an increase in the difference between the OTF of the captured PSFimage and the actual OTF. It should be noted that even if a generalfilter that reduces random noise, such as a median filter, frameintegration, or a low-pass filter, is caused to operate on the PSFimage, it is difficult to completely eliminate random noise from aregion of the PSF image where the luminance is low. Thus, it isdifficult to eliminate a change in the average luminance value of theentire PSF image.

As described above, in the case where a deteriorated image is restoredusing a captured PSF image, the average luminance value of the PSF imagechanges due to the influence of random noise (Gaussian noise), which isdifficult to be corrected, thereby causing a great change in thedirect-current component of the frequency components of the PSF.Consequently, a high-resolution restored image cannot be obtained. Sucha problem is revealed by the examination using the simulator in FIG. 4.

In view of the above, the following is a description of an imagingapparatus according to one aspect of the present invention, theapparatus being capable of solving the above problems.

Embodiment 1

The following is a description of Embodiment 1 of the present inventionwith reference to the drawings.

FIG. 1 is a block diagram showing a configuration of an imagingapparatus according to Embodiment 1 of the present invention. An imagingapparatus 10 includes an optical system 1, a PSF capturing unit 2 thatincludes a luminance reduction unit 3, a subject capturing unit 4, andan image restoration unit 5.

The optical system 1 captures a subject image. Specifically, the opticalsystem 1 includes a lens and an imaging device, for example. The opticalsystem 1 generates a PSF image I_psf(x, y) by capturing a point image ora subject image corresponding to a point image. Further, the opticalsystem 1 generates a subject image I_img(x, y) by capturing an arbitrarysubject image.

The PSF capturing unit 2 causes the optical system 1 to capture a pointimage or a subject image corresponding thereto in order to acquire a PSFcorresponding to the optical system 1, obtains the PSF image I_psf(x, y)from the optical system 1, and stores the image. Here, x represents animage position in the vertical direction in the image, whereas yrepresents an image position in the horizontal direction.

In other words, the PSF capturing unit 2 acquires PSF information. Here,PSF information is based on the PSF image I_psf(x, y) captured by theoptical system 1. Specifically, PSF information indicates the PSF imageI_psf(x, y) itself, for example. Alternatively, PSF information may beinformation obtained by converting the PSF image I_psf(x, y) from thespatial domain into the frequency domain, for example.

When there is known fixed value noise that does not change with timedepending on image positions (e.g., dark current noise, noise thatoccurs in predetermined lines or at predetermined pixel positions due tomanufacturing defects of the imaging device, or the like), the luminancereduction unit 3 subtracts a luminance value Nf(x, y) of the fixed valuenoise obtained in advance at each image position of the PSF imageI_psf(x, y) from the PSF image I_psf(x, y) as shown by Expression 2.Ir1_(—) psf(x,y)=I _(—) psf(x,y)−Nf(x,y)  (Expression 2)

Furthermore, the luminance reduction unit 3 subtracts a predeterminedluminance value Is1 from all positions of the PSF image Ir1_psf(x, y)obtained by subtracting the luminance value of the fixed value noise, asshown by Expression 3. Specifically, the luminance reduction unit 3subtracts a luminance value greater than the luminance value of thefixed value noise from the PSF image I_psf(x, y) through subtractionprocessing in accordance with Expressions 2 and 3.Ir2_(—) psf(x,y)=Ir1_(—) psf(x,y)−Is1  (Expression 3)

It should be noted that the luminance value is 0 at a position where theluminance value is negative in the PSF image Ir1_psf(x, y) obtained bysubtracting the luminance value of the fixed value noise. In otherwords, in the PSF image Ir1_psf(x, y) obtained by subtracting theluminance value of the fixed value noise, the luminance value at aposition where the luminance value is smaller than the lowest luminancesetting value is changed to the lowest luminance setting value. Here,the lowest luminance setting value is a value that indicates the lowestluminance among values that each indicate a luminance, and is 0 in thepresent embodiment.

Although Expressions 2 and 3 are described as separate expressions, theluminance reduction unit 3 may rationalize the calculation by performingExpressions 2 and 3 in the same step. The PSF capturing unit 2 outputsthe corrected PSF image Ir2_psf(x, y) obtained by subtracting theluminance value. The corrected PSF image Ir2_psf(x, y) is normalized asnecessary, and the result is output.

In this manner, the PSF capturing unit 2 subtracts, from PSFinformation, a correction luminance value greater by the luminance valueIs1 than the luminance value Nf of the fixed value noise that does notfluctuate with time using the luminance reduction unit 3, and outputscorrected PSF information obtained as a result of the subtraction.Specifically, the PSF capturing unit 2 subtracts the correctionluminance value from the entire region represented by the PSFinformation.

In other words, the PSF capturing unit 2 subtracts the correctionluminance value greater by the luminance value Is1 than the luminancevalue Nf of the fixed value noise from the luminance value of each pixelthat constitutes the PSF image I_psf(x, y). Furthermore, when thesubtraction result is smaller than the lowest luminance setting value,the PSF capturing unit 2 generates the corrected PSF image Ir2_psf(x, y)by correcting the subtraction result so as to match the lowest luminancesetting value.

Then, the PSF capturing unit 2 outputs corrected PSF information basedon the corrected PSF image Ir2_psf(x, y) generated in this manner. Thecorrected PSF information indicates the corrected PSF image Ir2_psf(x,y) itself, for example. Alternatively, the corrected PSF information maybe information obtained by converting the corrected PSF image Ir2_psf(x,y) from the spatial domain into the frequency domain, for example.

It should be noted that the reason for subtracting the luminance valueIs1 and the setting range of the luminance value Is1 will be describedbelow.

The subject capturing unit 4 stores subject images I_img(x, y) ofvarious subjects acquired by the optical system 1. The subject capturingunit 4 may perform compensation of the fixed value noise or noisecompensation processing such as median filtering on the subject imagesI_img(x, y), as necessary.

Specifically, the subject capturing unit 4 acquires a subject imageI_img(x, y) captured by the optical system 1, and outputs subjectinformation. Here, subject information is information based on theacquired subject image I_img(x, y). For example, subject information isinformation that indicates the subject image I_img(x, y) itself.Further, subject information may be information that indicates an imageobtained by performing various types of noise compensation processing onthe subject image I_img(x, y), for example. Also, subject informationmay be information obtained by converting the subject image I_img(x, y)or an image obtained as a result of performing various types of noisecompensation processing on the subject image I_img(x, y) from thespatial domain into the frequency domain, for example.

The image restoration unit 5 creates a restored image by performing animage restore operation such as Wiener filtering based on the correctedPSF image and the subject image. Specifically, the image restorationunit 5 performs a restore operation for restoring subject informationbased on the corrected PSF information and the subject information. Inother words, the image restoration unit 5 generates a restored imagehaving a higher resolution than that of the image indicated by thesubject information, by performing an image restore operation forcausing corrected PSF information to affect the subject information.

Specifically, the image restoration unit 5, for example, converts thecorrected PSF image Ir2_psf(x, y) indicated by the corrected PSFinformation and the subject image I_img(x, y) indicated by the subjectinformation from the spatial domain into the frequency domain, andcomputes a value of a frequency component at each frequency, therebygenerating a restored image.

It should be noted that the corrected PSF image Ir2_psf(x, y) may be animage once captured and corrected at the time of factory shipment,maintenance, or the like. Specifically, the image restoration unit 5 mayhave a storage means such as a memory, store in advance the correctedPSF image Ir2_psf(x, y) generated by the PSF capturing unit 2, andgenerate a restored image using the stored corrected PSF imageIr2_psf(x, y). In other words, the PSF capturing unit 2 need notnecessarily generate the corrected PSF image Ir2_psf(x, y) each time thesubject image I_img(x, y) is changed.

Further, the image restoration unit 5 may store frequency-domain datathat is the Fourier transform of the corrected PSF image Ir2_psf(x, y)according to an image restore algorithm, rationalization of calculation,or the like. In other words, the image restoration unit 5 may storecorrected PSF information.

Next is a description of various operations of the imaging apparatushaving the above configuration according to the present embodiment.

FIG. 10 shows flowcharts showing the operation of the above-describedimaging apparatus according to Embodiment 1 of the present invention.Specifically, (a) in FIG. 10 is a flowchart showing the flow ofcorrected PSF information generation processing. Further, (b) in FIG. 10is a flowchart showing the flow of, image restore processing. Asdescribed above, it is sufficient to perform the processing shown in (a)in FIG. 10 at least once prior to the processing shown in (b) in FIG.10, and the processing shown in the drawings need not necessarily beperformed in synchronization.

First is a description of the flowchart shown in (a) in FIG. 10.

The optical system 1 captures a PSF image I_psf(x, y) (S101). Next, theluminance reduction unit 3 subtracts the luminance value Nf(x, y) of thefixed value noise from the PSF image I_psf(x, y) in accordance withExpression 2, and thereby obtains a PSF image Ir1_psf(x, y) obtained asa result of subtracting the luminance value of the fixed value noise(S102). Furthermore, the luminance reduction unit 3 obtains a correctedPSF image Ir2_psf(x, y) by subtracting, in accordance with Expression 3,the luminance value Is1 to from the PSF image Ir1_psf(x, y) obtained asa result of subtracting the luminance value of the fixed value noise(S103). It should be noted that in the corrected PSF image Ir2_psf(x,y), a luminance value smaller than the lowest luminance setting value isreplaced with the lowest luminance setting value.

At last, the PSF capturing unit 2 normalizes the corrected PSF imageIr2_psf(x, y) obtained thereby, and outputs the result to the imagerestoration unit 5 (S104).

It should be noted that the luminance reduction unit 3 need notnecessarily perform processing in steps S102 and S103 in the statedorder, as described above. Specifically, the luminance reduction unit 3may perform processing in steps S102 and S103 as the processing in onestep by subtracting the sum of the luminance value Nf(x, y) of the fixedvalue noise and the luminance value Is1 from the PSF image I_psf(x, y).

It should be noted that fixed value noise subtraction processing in stepS102 need not necessarily be executed when the fixed value noise is verysmall, for instance.

Next is a description of the flowchart shown in (b) in FIG. 10.

The optical system 1 captures a subject image I_img(x, y) (S111). Next,the subject capturing unit 4 performs noise compensation processing onthe captured subject image I_img(x, y) (S112). At last, the imagerestoration unit 5 performs a restore operation based on the subjectimage I_img(x, y) on which noise compensation processing has beenperformed and the corrected PSF image Ir2_psf(x, y), thereby generatinga restored image (S113).

It should be noted that noise compensation processing in step S112 neednot necessarily be executed.

Next is a description of a reason for subtracting the luminance valueIs1 and the setting range of the luminance value Is1. In the followingdescription, a deteriorated image of the cuneal chart shown in (c) inFIG. 2 is used as a subject image I_img(x, y). Further, as a PSF imageIr1_psf(x, y) obtained by subtracting the luminance value is of thefixed value noise (hereinafter, also simply referred to as PSF imageIr1_psf(x, y)), an image obtained by adding Gaussian noise whosestandard deviation σ is 0.3% of the highest luminance value to the PSFimage in (b) in FIG. 2 is used (the luminance value at an image positionhaving a negative luminance value has already been corrected to “0”).

(a) in FIG. 11 shows a luminance distribution on lines that include theposition of the highest luminance value of the PSF image Ir1_psf(x, y).Since fixed value noise has been eliminated, a portion around theposition of the highest luminance value has a luminance distributionbased on the optical system 1 in FIG. 1, and the luminance value issubstantially “0” at positions distant from the position of the highestluminance.

(b) in FIG. 11 shows a luminance distribution of an enlarged portion inthe vicinity of the dashed line in (a) in FIG. 11. It can be seen thatthere is a slight luminance distribution (slight fluctuation in theluminance value) even at positions distant from the position of thehighest luminance value, due to the influence of Gaussian noise that israndomly distributed.

(c) in FIG. 11 shows an OTF that is the Fourier transform of the PSFimage Ir1_psf(x, y). As is clear from (c) in FIG. 11, the component at afrequency of 0 has significantly higher gain than that of otherfrequency components. A conceivable reason for this is a great increasein the average luminance value of the entire PSF image Ir1_psf(x, y) dueto Gaussian noise as described above.

Therefore, when an image restore operation is performed using the PSFimage Ir1_psf(x, y), the resolution of the restored image greatlydecreases due to an increase in the difference between the OTF in (c) inFIG. 11 and the actual OTF.

(a) in FIG. 12 shows a luminance distribution on lines including theposition of the highest luminance value of the corrected PSF imageIr2_psf(x, y) in the case where the predetermined luminance value Is1 issubtracted from all positions in the PSF image Ir1_psf(x, y) usingExpression 3. (b) in FIG. 12 shows a luminance distribution of anenlarged portion in the vicinity of the dashed line in (a) in FIG. 12.It can be seen that the slight luminance distribution at positionsdistant from the position of the highest luminance value due to theinfluence of Gaussian noise that is randomly distributed has beeneliminated.

(c) in FIG. 12 shows an OTF that is the Fourier transform of thecorrected PSF image Ir2_psf(x, y). It can be seen that the gain of thecomponent at a frequency of 0 has been improved, which is significantlyhigher compared with that of other frequency components in (c) in FIG.11, thereby obtaining a distribution closer to the actual OTFdistribution as in (b) in FIG. 8.

FIG. 13 shows restored images created by the image restoration unit 5performing an image restore operation using the corrected PSF imageIr2_psf(x, y). (a) in FIG. 13 shows a restored image in the case whereIs1 is 0, (b) in FIG. 13 shows a restored image in the case where Is1constitutes 0.5% of the highest luminance value of the PSF imageIr1_psf(x, y), and (c) in FIG. 13 shows a restored image in the casewhere Is1 constitutes 1% of the highest luminance value of the PSF imageIr1_psf(x, y).

It can be seen that the resolution of the restored image in (b) in FIG.13 has been improved compared with that in (a) in FIG. 13, and theresolution of the restored image in (c) in FIG. 13 has been furtherimproved compared with that in (b) in FIG. 13.

FIG. 14 shows a change in the resolution of a restored image when Is1 ischanged. In the graph shown in FIG. 14, the vertical axis represents theresolution measured using the resolution measurement tool HYRes3.1distributed from CIPA, whereas the horizontal axis represents theproportion of Is1 to the highest luminance value of the PSF imageIr1_psf(x, y).

As is clear from FIG. 14, the resolution improves when Is1 is 0.3% orhigher of the highest luminance value of the PSF image Ir1_psf(x, y).Specifically, in the PSF image, the ratio between the luminance valuebased on the actual PSF and the luminance value of noise exerts greatinfluence on the resolution, and the resolution improves when Is1 is setto the standard deviation σ of Gaussian noise included in the PSF imageor higher. Thus, assuming that random noise included in the PSF image isGaussian noise, Is1 is preferably equal to or greater than the standarddeviation σ of the Gaussian noise.

Furthermore, if Is1 is set to 0.6% or more of the highest luminancevalue of the PSF image Ir1_psf(x, y), it is possible to obtain theresolution equivalent to that in the case where there is no Gaussiannoise. The resolution of a restored image stops improving if Is1 ishigher than 30% of the highest luminance value of the PSF imageIr1_psf(x, y), which is not shown in the drawings. In other words, theresolution of a restored image starts decreasing if the proportion ofIs1 to the highest luminance value of the PSF image Ir1_psf(x, y) ishigher than 30%. Therefore, it is preferable that the proportion of Isto the highest luminance value of the PSF image be a value from 0.3% to30%.

In this manner, although information on a region having a low luminancewill be lost from the actual PSF luminance distribution by performingthe subtraction of Expression 2, a decrease in the resolution due tonoise that is superimposed on the PSF image has more influence than adecrease in the resolution of the restored image due to that lost, andthus correction of the PSF image by performing the subtraction ofExpression 2 can improve the resolution of the restored image.

It should be noted that Gaussian noise (random noise) is considered tobe caused by current noise or the like, and is noise whose occurrenceposition (pixel) randomly changes. However, Gaussian noise has a featurethat the distribution of luminance values does not greatly change withtime. Therefore, the standard deviation σ of Gaussian noise can bespecified in a comparatively stable manner by approximating thedistribution of luminance values of a black image captured in, forinstance, a darkroom using a Gaussian distribution.

As described above, according to the imaging apparatus 10 according toEmbodiment 1 of the present invention, when an image restore operationis performed, even in the case where unnecessary luminance (especially,random noise that fluctuates with time) of a captured PSF image is high,a correction luminance value greater than the luminance value of thefixed value noise is subtracted from the PSF image so as to correct theaverage luminance value to an appropriate value, thereby obtaining moreaccurate PSF information for restoring an image and enablinghigh-resolution image restoration.

An example of a general method of reducing random noise is a methodusing a median filter, frame integration, or a low-pass filter. Even ifnoise is reduced from the PSF image using such a median filter, frameintegration, or a low-pass filter, it is difficult to completelyeliminate random noise from a region of the PSF image where theluminance is low, and thus it is difficult to eliminate a change in theaverage luminance value of the PSF image. Therefore, even if noiseelimination processing using a median filter, frame integration, or alow-pass filter is performed, it is not possible to restore adeteriorated image to a high-resolution image.

On the other hand, in the present embodiment, as shown by Expressions 2and 3, a luminance value greater than the luminance value of the fixedvalue noise is forcibly subtracted from the PSF image. Therefore, achange in the average luminance value in the PSF image can beeliminated, thereby enabling restoration of a deteriorated image to ahigh-resolution image.

It should be noted that generally, when random noise is eliminated froma captured image, a method that does not change the average luminancevalue (e.g., median filtering, frame integration, or low-pass filtering)is used, and a method of forcibly subtracting a predetermined luminancevalue as in the present embodiment is not used. This is because asubject image having a comparatively low luminance in the captured imagedisappears if the predetermined luminance value is forcibly subtractedfrom the captured image, which will not allow restoration thereof. Onthe other hand, in the present embodiment, most of the subject imagewill not disappear since a luminance value is subtracted from the PSFimage, and thus practical problems will not occur.

It should be noted that the luminance reduction unit 3 need notnecessarily subtract the luminance value of the fixed value noise inaccordance with Expression 2. In other words, it is sufficient for theluminance reduction unit 3 to subtract the luminance value of the fixedvalue noise from the luminance value of the PSF image as necessary, andthus it goes without saying that subtraction of the luminance value ofthe fixed value noise is not required.

It should be noted that in Expression 3 described above, although theluminance value Is1 is a constant value independent of a pixel position,the luminance value Is1 need not necessarily be a constant valueindependent of a pixel position. Specifically, it is sufficient that theluminance value Is1 is a value greater than 0 (preferably, theproportion of the value to the highest luminance value is 0.3% to 30%),and the luminance value Is1 may be a value different for each pixel.

It should be noted that although Embodiment 1 of the present inventionhas been described using an example in which the PSF has a symmetricalluminance distribution with the image position having the highestluminance being the center as shown in (b) in FIG. 2, it goes withoutsaying that the present invention is applicable to an optical systemhaving an asymmetrical PSF luminance distribution.

Embodiment 2

The configuration of the imaging apparatus 10 according to Embodiment 2of the present invention is also shown in FIG. 1 as in the case ofEmbodiment 1. In the present embodiment, the operation of the luminancereduction unit 3 differs from that in Embodiment 1 in the block diagramof FIG. 1. Other operations are the same as those in Embodiment 1, andthus description thereof is omitted.

If there is fixed value noise that does not change with time dependingon image positions (e.g., dark current noise, noise that occurs inpredetermined lines or at predetermined pixel positions due tomanufacturing defects of the imaging device, or the like), the luminancereduction unit 3 subtracts, from the PSF image I_psf(x, y), a luminancevalue Nf(x, y) of the fixed value noise obtained in advance at eachimage position as shown by Expression 2.

Furthermore, the luminance reduction unit 3 subtracts the luminancevalue at an image position having a luminance value smaller than apredetermined luminance value Is2 among image positions of the PSF imageIr1_psf(x, y) obtained by subtracting the luminance value of the fixedvalue noise, thereby correcting the luminance value to “0”, as shown byExpression 4. Specifically, the luminance reduction unit 3 subtracts aluminance value greater than the luminance value of the fixed valuenoise at an image position where the luminance value of the PSF imageI_psf(x, y) is smaller than the sum of the luminance value Nf(x, y) ofthe fixed value noise and the predetermined luminance value Is2 byperforming subtraction processing in accordance with Expressions 2 and4.If (Ir1_(—) psf(x,y)<Is2)Ir2_(—) psf(x,y)=0  (Expression 4)

It should be noted that although Expressions 2 and 4 are described asseparate expressions, the luminance reduction unit 3 may rationalize thecalculation by replacing Is2 with the sum of Is2 and Nf(x, y) andperforming the calculation shown by Expressions 2 and 4 at the sametime.

In other words, the PSF capturing unit 2 subtracts, from PSFinformation, a correction luminance value greater by the luminance valueIs2 than the luminance value Nf of the fixed value noise that does notfluctuate with time using the luminance reduction unit 3, and outputscorrected PSF information obtained as a result of the subtraction.Specifically, the PSF capturing unit 2 subtracts the correctionluminance value only from a region that is included in the entire regionrepresented by the PSF information and has a smaller luminance valuethan the correction luminance value.

Thus, the PSF capturing unit 2 subtracts the correction luminance valuegreater by the luminance value Is2 than the luminance value Nf of thefixed value noise only from the luminance value of a pixel that issmaller than the correction luminance value among a plurality of pixelsthat constitute the PSF image I_psf(x, y). Furthermore, if thesubtraction result is smaller than the lowest luminance setting value,the PSF capturing unit 2 generates a corrected PSF image Ir2_psf(x, y)by correcting the subtraction result so as to match the lowest luminancesetting value.

In other words, the PSF capturing unit 2 replaces the luminance value ofa region that is included in the entire region represented by to the PSFinformation and has a smaller luminance value than the correctionluminance value with the lowest luminance setting value. Thus, the PSFcapturing unit 2 replaces the luminance value of a pixel that is smallerthan the sum of the luminance value Nf(x, y) of the fixed value noiseand the luminance value Is2 with the lowest luminance setting value,among a plurality of pixels that constitute the PSF image I_psf(x, y).

In this manner, the PSF capturing unit 2 generates the corrected PSFimage Ir2_psf(x, y) by correcting the luminance value of a pixel that issmaller than the correction luminance value to the lowest luminancesetting value among a plurality of pixels that constitute the PSF imageI_psf(x, y).

Next is a description of various operations of the imaging apparatushaving the above configuration according to the present embodiment.

FIG. 15 shows flowcharts showing operation of the imaging apparatusaccording to Embodiment 2 of the present invention described above.Specifically, (a) in FIG. 15 is a flowchart showing the flow ofcorrected PSF information generation processing. Further, (b) in FIG. 15is a flowchart showing the flow of image restore processing. It shouldbe noted that the steps where the same processing as that in FIG. 10 isperformed are given identical numerals in FIG. 15, and descriptionthereof is omitted.

After performing fixed value noise subtraction processing, the luminancereduction unit 3 subtracts the correction luminance value only from aregion that is included in the entire region represented by the PSFinformation and has a smaller luminance value than the correctionluminance value, and outputs corrected PSF information obtained as aresult of the subtraction (S201).

In this manner, the PSF capturing unit 2 generates a corrected PSF imageIr2_psf(x, y) by correcting the luminance value of a pixel that issmaller than the correction luminance value to the lowest luminancesetting value in the PSF image I_psf(x, y).

Next is a description of a setting range of the luminance value Is2. Inthe following description, a deteriorated image of the cuneal chartshown in (c) in FIG. 2 is used as a subject image I_img(x, y). Further,as a PSF image Ir1_psf(x, y) obtained by subtracting the luminance valueof the fixed value noise (hereinafter, also simply referred to as PSFimage Ir1_psf(x, y)), an image obtained by adding Gaussian noise whosestandard deviation σ is 0.3% of the highest luminance value to the PSFimage in (b) in FIG. 2 is used (the luminance value at an image positionhaving a negative luminance value has already been corrected to “0”).

(a) in FIG. 16 shows a luminance distribution on lines including aposition of the highest luminance value of the corrected PSF imageIr2_psf(x, y) when the luminance value at a position having a smallerluminance value than the predetermined luminance value Is2 in the PSFimage Ir1_psf(x, y) has been corrected to “0” using Expression 4. (b) inFIG. 16 shows a luminance distribution of an enlarged portion in thevicinity of the dashed line in (a) in FIG. 16. It can be seen that aslight luminance distribution at positions distant from the position ofthe highest luminance value due to the influence of Gaussian noise thatis randomly distributed has been eliminated.

(c) in FIG. 16 shows an OTF that is the Fourier transform of thecorrected PSF image Ir2_psf(x, y). It can be seen that the gain of thecomponent at a frequency of 0, which is significantly higher comparedwith that of other frequency components in (c) in FIG. 11, has beencorrected, thereby obtaining a distribution closer to the actual OTFdistribution as in (b) in FIG. 8.

FIG. 17 shows restored images created by the image restoration unit 5 inFIG. 1 performing an image restore operation using the corrected PSFimage Ir2_psf(x, y). (a) in FIG. 17 shows a restored image in the casewhere Is2 is 0, (b) in FIG. 17 shows a restored image in the case whereIs2 constitutes 0.75% of the highest luminance value of the PSF imageIr1_psf(x, y), and (c) in FIG. 17 shows a restored image in the casewhere Is2 constitutes 1.5% of the highest luminance value of the PSFimage Ir1_psf(x, y).

It can be seen that the resolution of the restored image in (b) in FIG.17 has been improved compared with that in (a) in FIG. 17, and theresolution of the restored image in (c) in FIG. 17 has been furtherimproved compared with that in (b) in FIG. 17.

FIG. 18 shows a change in the resolution of a restored image when Is2 ischanged. In the graph shown in FIG. 18, the vertical axis represents theresolution measured using the resolution measurement tool HYRes3.1distributed from CIPA, whereas the horizontal axis represents theproportion of Is2 to the highest luminance value of the PSF imageIr1_psf(x, y).

As is clear from FIG. 18, the resolution improves when Is2 is set to0.5% or more of the highest luminance value of the PSF image Ir1_psf(x,y). Specifically, in the PSF image, the ratio between the luminancevalue based on the actual PSF and the luminance value of noise exertsgreat influence on the resolution, and the resolution improves when Is2is set to a value about 1.6 (=0.5/0.3) times or more greater than thestandard deviation σ of Gaussian noise included in the PSF image. Thus,assuming that random noise included in the PSF image is Gaussian noise,Is2 is preferably 1.6 times or more greater than the standard deviationof the Gaussian noise.

Furthermore, when Is2 is set to 1% or more of the highest luminancevalue of the PSF image Ir1_psf(x, y), it is possible to obtain theresolution equivalent to that in the case where there is no Gaussiannoise. The resolution of a restored image stops improving if Is2 ishigher than 40% of the highest luminance value of the PSF imageIr1_psf(x, y), which is not shown in the drawings. In other words, theresolution of a restored image starts decreasing if the proportion ofIs2 to the highest luminance value of the PSF image Ir1_psf(x, y) ishigher than 40%. Thus, it is preferable that the proportion of Is2 tothe highest luminance value of the PSF image be a value from 0.5% to40%.

In this manner, although information on a region having a low luminancein the actual PSF luminance distribution will be lost by performing thecorrection of Expression 4, a decrease in the resolution due to noisethat is superimposed on the PSF image has more influence than a decreasein the resolution of the restored image due to that lost, and thuscorrection of the PSF image using Expression 4 can improve theresolution of the restored image.

As described above, according to the imaging apparatus 10 according toEmbodiment 2 of the present invention, when an image restore operationis performed, even in the case where unnecessary luminance (especially,random noise that fluctuates with time) of a captured PSF image is high,a luminance value greater than the fixed value noise is subtracted fromthe PSF image so as to correct the average luminance value to anappropriate value, thereby obtaining more accurate PSF information forrestoring an image and enabling high-resolution image restoration.

It should be noted that although the imaging apparatus 10 according tothe present embodiment corrects the luminance value at an image positionhaving a luminance value smaller than Is2 in the PSF image Ir1_psf asshown by Expression 4, even when the luminance value at an imageposition having a luminance value equal to or smaller than Is2 iscorrected as shown by Expression 5, substantially the same effects canbe obtained.If (Ir1_(—) psf(x,y)≦Is2)Ir2_(—) psf(x,y)=0  (Expression 5)

It should be noted that the luminance reduction unit 3 need notnecessarily subtract the luminance value of the fixed value noise, whichis shown by Expression 2. In other words, it is sufficient for theluminance reduction unit 3 to subtract the luminance value of the fixedvalue noise from the luminance value of the PSF image as necessary, andthus it goes without saying that subtraction of the luminance value ofthe fixed value noise is not required.

It should be noted that in Expression 4 described above, although theluminance value Is2 is a constant value independent of a pixel position,Is2 need not necessarily be a constant value independent of a pixelposition. Specifically, it is sufficient that the luminance value Is2 isa value greater than 0 (preferably, the proportion of the value to thehighest luminance value is 0.5% to 40%), and the luminance value Is2 maybe a value different for each pixel.

It should be noted that although Embodiment 2 of the present inventionhas been described using an example in which the PSF has a symmetricalluminance distribution with the image position having the highestluminance value being the center as shown in (b) in FIG. 2, it goeswithout saying that the present embodiment is applicable to an opticalsystem having an asymmetrical PSF luminance distribution.

Although the above is a description of the imaging apparatus 10according to an aspect of the present invention based on theembodiments, the present invention is not limited to those embodiments.The scope of the present invention includes various modifications to theembodiments that may be conceived by those skilled in the art or formsconstructed by combining constituent elements in different embodiments,which do not depart from the essence of the present invention.

For example, although the imaging apparatus 10 includes the PSFcapturing unit 2 in Embodiments 1 and 2 described above, the imagingapparatus 10 need not necessarily include the PSF capturing unit 2.Specifically, it is sufficient for the imaging apparatus 10 to storecorrected PSF information generated in advance. Even in this case, theimaging apparatus 10 can restore a high-resolution subject image sincethe imaging apparatus 10 can perform a restore operation for restoringsubject information based on the corrected PSF information stored inadvance and the subject information.

Further, a part or all of the constituent elements included in theimaging apparatus 10 in Embodiments 1 and 2 described above may beconstituted by a single system large scale integration (LSI). Forexample, the imaging apparatus 10 may be constituted by a system LSThaving the PSF capturing unit 2, the subject capturing unit 4, and theimage restoration unit 5.

The system LSI is a super multi-function LSI that is manufactured byintegrating multiple components in one chip, and is specifically acomputer system configured so as to include a microprocessor, a readonly memory (ROM), a random access memory (RAM), and so on. A computerprogram is stored in the RAM. The system LSI accomplishes its functionsthrough the operation of the microprocessor in accordance with thecomputer program.

It should be noted that although a system LSI is mentioned here, theintegrated circuit can also be called an IC, an LSI, a super LSI, and anultra LSI, depending on the difference in the degree of integration.Furthermore, the method of circuit integration is not limited to LSIs,and implementation through a dedicated circuit or a general-purposeprocessor is also possible. A field programmable gate array (FPGA) thatallows programming after LSI manufacturing or a reconfigurable processorthat allows reconfiguration of the connections and settings of thecircuit cells inside the LSI may also be used.

In addition, depending on the emergence of circuit integrationtechnology that replaces LSI due to progress in semiconductor technologyor other derivative technology, it is obvious that such technology maybe used to integrate the function blocks. Possibilities in this regardinclude the application of biotechnology and the like.

Further, the present invention can be implemented, not only as animaging apparatus that includes such characteristic processing units asthose described above, but also as an image restoration method having,as steps, the characteristic processing units included in such animaging apparatus. Furthermore, the present invention can also berealized as a computer program that causes a computer to execute thecharacteristic steps included in the image restoration method. Inaddition, it goes without saying that such a computer program can bedistributed via a computer-readable recording medium such as a compactdisk read-only memory (CD-ROM) or via a communication network such asthe Internet.

The present invention is useful in imaging apparatuses in general thatcapture a subject image using an optical system, such as a digital stillcamera, a digital video camera, a mobile telephone camera, a monitoringcamera, a medical camera, a telescope, a microscope, a vehicle-installedcamera, a stereo ranging camera, a stereoscopic video shootingmulti-lens camera, a light beam space capture camera for free-viewpointvideo creation, an extended depth of field camera (EDOF), and a camerausing flexible depth of field (FDOF) photography.

REFERENCE SIGNS LIST

-   -   1 Optical system    -   2 PSF capturing unit    -   3 Luminance reduction unit    -   4 Subject capturing unit    -   5 Image restoration unit    -   10 Imaging apparatus    -   101 Deteriorated-image noise adding unit    -   102 PSF-image noise adding unit    -   103 Image restore operation unit

The invention claimed is:
 1. An imaging apparatus comprising: an opticalsystem; a point spread function (PSF) capturing unit configured toacquire PSF information captured by the optical system, and outputcorrected PSF information; a subject capturing unit configured toacquire subject information captured by the optical system, and outputthe acquired subject information; and an image restoration unitconfigured to perform a restore operation for restoring the subjectinformation, based on the corrected PSF information and the subjectinformation, wherein the PSF capturing unit is configured to subtract acorrection luminance value from a luminance value of a PSF imageindicated by the PSF information, and output the corrected PSFinformation obtained as a result of the subtraction, the correctionluminance value being greater by a luminance value Is than a luminancevalue Nf of fixed value noise that does not fluctuate with time.
 2. Theimaging apparatus according to claim 1, wherein the PSF capturing unitis configured to subtract the correction luminance value from an entireregion represented by the PSF information.
 3. The imaging apparatusaccording to claim 2, wherein assuming that random noise included in thePSF information is Gaussian noise, the luminance value Is is equal to orgreater than a standard deviation of the Gaussian noise.
 4. The imagingapparatus according to claim 2, wherein the luminance value Is is avalue from 0.3% to 30% of a highest luminance value of the PSFinformation.
 5. The imaging apparatus according to claim 1, wherein thePSF capturing unit is configured to subtract the correction luminancevalue only from a region having a luminance value equal to or smallerthan the correction luminance value, or a region having a luminancevalue smaller than the correction luminance value, the regions beingincluded in the entire region represented by the PSF information.
 6. Theimaging apparatus according to claim 5, wherein assuming that randomnoise included in the PSF information is Gaussian noise, the luminancevalue Is is 1.6 times or more greater than a standard deviation of theGaussian noise.
 7. The imaging apparatus according to claim 5, whereinthe luminance value Is is a value from 0.5% to 40% of a highestluminance value of the PSF information.
 8. An imaging apparatuscomprising: an optical system; a subject capturing unit configured toacquire subject information captured by the optical system, and outputthe acquired subject information; and an image restoration unitconfigured to perform a restore operation for restoring the subjectinformation based on corrected point spread function (PSF) informationstored in advance and the subject information, wherein the corrected PSFinformation is information obtained by subtracting a correctionluminance value from a luminance value of a PSF image indicated by PSFinformation captured by the optical system, the correction luminancevalue being greater by a luminance value Is than a luminance value Nf offixed value noise that does not fluctuate with time.
 9. An imagerestoration method comprising: acquiring point spread function (PSF)information captured by an optical system, and outputting corrected PSFinformation; acquiring subject information captured by the opticalsystem, and outputting the acquired subject information; and performinga restore operation for restoring the subject information, based on thecorrected PSF information and the subject information, wherein in theacquiring of PSF information, a correction luminance value is subtractedfrom a luminance value of a PSF image indicated by the PSF information,and the corrected PSF information obtained as a result of thesubtraction is output, the correction luminance value being greater by aluminance value Is than a luminance value Nf of fixed value noise thatdoes not fluctuate with time.
 10. A non-transitory computer-readablerecording medium storing a program for causing a computer to execute theimage restoration method according to claim 9.